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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )^ Responsive to communication(s) filed on 15 December 2003 . 
2a)D This action is FINAL. 2b)S This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-25 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) E3 Claim(s) 1-25 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. §§ 119 and 120 

1 2)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 1 9(a)-(d) or (f). 
a)C3 All b)D Some*c)D None of: 

1 .□ Certified copies of the priority documents have been received. 
20 Certified copies of the priority documents have been received in Application No. 



30 Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

13) Q Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application) 

since a specific reference was included in the first sentence of the specification or in an Application Data Sheet. 
37 CFR 1.78. 

a) □ The translation of the foreign language provisional application has been received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 since a specific 

reference was included in the first sentence of the specification or in an Application Data Sheet. 37 CFR 1 .78. 
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1) □ Notice of References Cited (PTO-892) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) D Information Disclosure Statement(s) (PTO-1449) Paper No(s), 



4) Q Interview Summary (PTO-413) Paper No(s). 

5) □ Notice of Informal Patent Application (PTO-152) 

6) □ Other: 
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DETAILED ACTION 



1 . This office action is responsive to the following communications: Application, 
filed 03/27/2001 ; Amendment A, filed 06/30/2003; Amendment B, filed 1 1/10/2003; 
Amendment C, filed 12/15/2003. 

2. Claims 1-24 are pending in this application. Claims 1 and 13 are independent 
claims. Claim 25 has been added. 

3. The present title of this application is "Augmented Reality Guided Instrument 
Positioning with Modulated Guiding Graphics" (as originally filed). 



4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claims 1-24 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Fukunaga, U.S. Patent Number 6,346,940 B1, "Virtualized Endoscope System", class 
345/427, 02/2002, filed 02/1998. 

As per independent claim 1, a method for augmented reality guided instrument 
positioning, comprising the steps of: determining a graphics guide for positioning an 
instrument; Fukunaga discloses guiding marker to indicate direction of the instrument, 
virtual endoscope, col. 8, lines 1-10; and rendering the graphics guide such that an 



Claim Rejections ■ 35 CISC § 103 
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appearance of at least one portion of the graphics guide is modulated with respect to at 
least one of space and time. 

However, it is noted that Fukunaga fails to disclose explicitly modulating the 
graphics guide appearance. 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention of Fukunaga preparing guiding marker that the image control unit processing 
the guiding marker for display at a desired position on the image, as the virtual 
endoscope is moved about the image the graphics guide would be changed to 
represent the portion of the image the endoscope is occupying. 

With respect to dependent claim 2, rendering step comprises the step of varying 
a transparency of the at least one portion of the graphics guide with respect to other 
portions of the graphics guide to provide a substantially unobstructed view through the 
at least one portion of the graphics guide . . . 

Fukunaga discloses semitransparent image so that portions of the image can be 
viewed unobstructed, col. 11, lines 44-67. 

However, it is noted that Fukunaga fails to disclose a transparent guide marker. I 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to include not only semi-transparent images, but also markers to avoid 
obstructing portions of the image, which is what Fukunaga wishes to accomplish. 

With respect to dependent claim 3, varying a transparency of the at least one 
portion of the graphics guide during pre-defined time intervals to provide a substantially 
unobstructed view through the at least one portion of the graphics guide . . . 
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Fukunaga discloses an adjustable degree of transparency, col. 7, lines 59-64. 

However, it is noted that Fukunaga fails to disclose a transparent guide marker. 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to include not only semi-transparent images, but also markers to avoid 
obstructing portions of the image, which is what Fukunaga wishes to accomplish. 

With respect to dependent claim 4, varying a transparency of each of a plurality 
of portions of the graphics guide during at least one predefined time interval to provide a 
substantially unobstructed view through each of the plurality of portion to at least a 
portion of the instrument . . . 

Fukunaga discloses an adjustable degree of transparency, col. 7, lines 59-64. 

However, it is noted that Fukunaga fails to disclose a transparent guide marker. 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to include not only semi-transparent images, but also markers to avoid 
obstructing portions of the image. 

With respect to dependent claim 5, plurality of portions are consecutive. 
Fukunaga discloses guiding markers as a path to take, col. 8, lines 5-10. 

With respect to dependent claim 6, varying a transparency of the at least one 
portion of the graphics guide such that the at least one portion repeatedly switches 
between transparent and less transparent. Fukunaga discloses thinning pixels based 
on a preset degree of semi transparency adjusted by the operator, col. 1 1 , lines 44-57. 
However, it is noted that Fukunaga fails to disclose a transparent guide marker. It 
would have been obvious to one of ordinary skill in the art at the time of the invention to 
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include not only semi-transparent images, but also guide markers to avoid obstructing 
portions of the image. 

With respect to dependent claim 7, constructing the graphics guide as a line, and 
rendering step comprises the step of modulating a transparency of the line with respect 
to time so that the line repeatedly fades in and out of view to provide a substantially 
unobstructed view . . . 

Fukunaga discloses the guiding marker used to indicate the direction could be a 
path, arrows, etc, col. 8, lines 1-10. 

However, it is noted that Fukunaga fails to disclose a transparent line. It would 
have been obvious to one of ordinary skill in the art at the time of the invention to 
include not only semi-transparent images, but also markers to avoid obstructing portions 
of the image. 

With respect to dependent claim 8, constructing the graphics guide as a line, and 
rendering step comprises the step of modulating a transparency of portions of the line 
so that at least a portion of the instrument is substantially unobstructed . . . 

Fukunaga discloses the guiding marker used to indicate the direction could be a 
path, arrows, etc, col. 8, lines 1-10. However, it is noted that Fukunaga fails to disclose 
a transparent line. 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to include not only semi-transparent images, but also markers to avoid 
obstructing portions of the image. 
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With respect to dependent claim 9, rendering step comprises the step of 
modulating a transparency of portions of the line with respect to time and space so that 
at least a portion of the instrument is substantially unobstructed . . . during pre-defined 
time intervals. 

Fukunaga discloses semitransparent image so that portions of the image can be 
viewed unobstructed, col. 11, lines 44-67. However, it is noted that Fukunaga fails to 
disclose a transparent line. 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to include not only semi-transparent images, but also markers to avoid 
obstructing portions of the image. 

With respect to dependent claim 10, constructing the graphics guide as a cylinder 
. . . rendering step comprises the step of modulating a transparency of the cylinder with 
respect to time so that the cylinder repeatedly fades in and out of view . . . 

Fukunaga discloses typical examples of guiding markers. However, it is noted 
that Fukunaga fails to disclose a transparent cylinder. It would have been obvious to 
one of ordinary skill in the art at the time of the invention to include not only semi- 
transparent images, but also graphic markers to avoid obstructing portions of the image. 

With respect to dependent claim 1 1 , modulating a transparency of portions of the 
cylinder so that at least a portion of the instrument is substantially unobstructed . . . 
Fukunaga discloses semitransparent image sot that portions of the image can be 
viewed unobstructed, col. 11, lines 44. However, it is noted that Fukunaga fails to 
disclose a transparent cylinder. It would have been obvious to one of ordinary skill in 
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the art at the time of the invention to include not only semi-transparent images, but also 
graphic guide markers to avoid obstructing portions of the image. 

With respect to dependent claim 12, rendering step comprises the step of 
modulating a transparency of portions of the cylinder with respect to time and space so 
that at least a portion of the instrument is substantially unobstructed . . . during pre- 
defined time intervals. 

Fukunaga discloses an operation path history that registers the path position and 
view direction each time the instrument changes to reproduce the image on the display 
unit, col. 8, lines 32-47. 

However, it is noted that Fukunaga fails to disclose a transparent line. It would 
have been obvious to one of ordinary skill in the art at the time of the invention to 
include not only semi-transparent images, but graphic also guide markers to avoid 
obstructing portions of the image. 

As per independent claim 13 and dependent claims 14-24, they are rejected 
based upon similar rational as above independent claim 1 and dependent claims 2-12 
respectively. 

As per independent claim 25, it is rejected based upon similar rational as above 
dependent claim 2. 

Response to Arguments 

6. Applicant's arguments filed 06/30/2003 have been fully considered but they are 
not persuasive. 
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Applicant argues that Fukunaga fails to disclose a graphics guide for positioning 
an instrument. Fukunaga discloses an endoscope tip state computation unit for display 
of an indicator image and the endoscope tip is the orientation, i.e. positioned, of the 
endoscope, i.e. instrument, in relation to the body and view direction, viewpoint position, 
and reference direction, col. 7, lines 12-26, and further discloses the endoscope tip is 
prepared for display either inside or outside depending on the viewpoint position, col. 7, 
lines 27-39, therefore the endoscope tip state computation unit performs the function of 
positioning an instrument. Fukunaga discloses guiding markers to indicate the three- 
dimensional coordinate positions within the image, col. 8, lines 1-10. Applicant argues 
that Fukunaga fails to disclose the graphics guide is modulated with respect to one of 
space and time. Fukunaga discloses an operation path history in which the viewpoint 
positions are indicated and the shortest path connecting the viewpoint positions is 
determined and the path history stores the images on the main display and sub-display, 
col. 8, lines 32-57. 

Applicant further discloses that Fukunaga fails to disclose the varying portion of 
the endoscope image and varying a transparency of the graphics guide. Fukunaga 
discloses semitransparent image so that portions of the image can be viewed 
unobstructed, col. 1 1 , lines 44-67. Fukunaga discloses masking for superimposing 
images and adjusting the pattern of the masking image to change the degree of one 
image through another, col. 12, lines 10-55 and discloses the image are varied. It is 
inherent that a variation of intensity of an image may take place over a time or a 
distance. Applicant argues that Fukunaga fails to disclose a transparency of the portion 
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of the graphics during pre-defined time intervals. Fukunaga discloses recorded 
simulation images, col. 5, lines 46-52, therefore portions of the image, which are viewed 
unobstructed, are transparent during pre-defined time intervals during the simulation. 

Applicant argues that Fukunaga fails to disclose modulating a transparency and 
a rendering device modulates a transparency. Fukunaga discloses varying, i.e. 
modulating, intensities of color, and a transparency parameter, col. 12, lines 10-25, 
therefore modulation is performed on the transparent color by the variation of the 
intensity of the color data. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Motilewa A. Good-Johnson whose telephone number is 
(703) 305-3939. The examiner can normally be reached on Monday - Friday 8:30 AM - 
5:00 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mike Razavi can be reached on (703) 305-4713. The fax phone numbers 
for the organization where this application or proceeding is assigned are (703) 872-9314 
for regular communications and (703) 872-9314 for After Final communications. 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the receptionist whose telephone number is (703) 306- 
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